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BY  THE  COMPTROLLER  GENERAL 

Report  To  The  Chairman,  Committee  On  ( 
Government  Operations 
House  Of  Representatives 

OF  THE  UNITED  STATES _ 

Air  Force  Progress  In  Implementing  The 

Phase  IV  Base  Level  Computer  Replacement  Program 

.  I 

N 

The  Air  Force  has  designed  a  program  -called 
Phase  IV-to  replace  computers  at  1 1 8  air  bases 
around  the  world  by  July  1 985.  Because  of  difficul¬ 
ties  in  converting  the  old  software  to  run  on  the  new 
computers  and  in  demonstrating  that  the  new  com¬ 
puters  can  meet  the  bases'  expected  workload,  the 
Air  Force  is  6  months  behind  schedule  in  imple¬ 
menting  Phase  IV  at  its  first  site.  According  to  the 
Air  Force,  this  delay  will  increase  overall  costs  by 
about  $6  million.  However,  the  Air  Force  expects  to 
recover  from  the  delay  by  March  1985.  Although 
every  effort  should  be  made  to  stay  on  schedule  and 
to  avoid  increased  costs,  GAO  believes  that  overall 
quality  should  not  be  compromised.  The  converted 
software  and  computers  must  be  thoroughly  tested 
before  being  implemented  at  other  bases. 

According  to  the  Air  Force,  a  1 980  redirection  of 
Phase  IV  has  produced  significant  savings.  The  Air 
Force  estimates  that,  as  a  result  of  Phase  IV  and 
refinements  made  incident  to  the  Phase  IV  redirec¬ 
tion,  total  acquisition  and  operation  savings  over 
the  next  20  years  will  be  $1.8  billion.  By  following 
through  with  commitments  made  to  the  House 
Committee  on  Government  Operations  in  1 980,  the 
Air  Force  can  realize  additional  savings. 

The  Assistant  Secretary  of  the  Air  Force  (Financial  1 

Management)  is  acting  on  issues  raised  in  this  |  j  [  ’ 

report.  *  I 
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The  Honorable  Jack  Brooks 
Chairman,  Committee  on 
Government  Operations 
House  of  Representatives 

Dear  Mr .  Chairman : 

In  response  to  your  January  13,  1983,  request  (see  app.  II) 
and  subsequent  discussions  with  your  office,  we  have  completed  a 
followup  review  of  the  Air  Force's  Phase  IV  Base  Level  Computer 
Replacement  Program. 

Phase  IV — the  largest  computer  acquisition  ever  attempted 
by  the  government— is  designed  to  replace  existing  computers  at 
118  air  bases  around  the  world.  The  software  operating  on  the 
current  computers  will  be  converted  to  operate  on  the  new  compu¬ 
ters.  The  Air  Force's  purchase  of  the  new  computers  installed 
at  each  base  is  contingent  upon  successful  demonstration  of  the 
hardware  and  converted  software.  After  the  Air  Force  completes 
a  detailed  analysis  of  user  information  needs,  scheduled  for 
1985,  it  will  redesign  the  converted  software.  Over  the  next  20 
years.  Phase  IV  acquisition  and  operation  costs  will  total  about 
§5  billion. 

Under  Phase  IV,  the  Air  Force  originally  planned  to  buy 
2  computers  for  most  of  its  bases — or  a  total  of  227  computers. 
At  your  committee's  hearings  in  October  1979,  we  testified  that 
this  approach  was  unnecessarily  expensive  and  restricted  compe¬ 
tition.  Subsequently,  the  Air  Force  made  specific  commitments 
to  your  committee  to  redirect  Phase  IV  and  to  (1)  reduce  the 
costs  of  computer  equipment,  personnel,  and  facilities;  (2)  sig¬ 
nificantly  improve  day-to-day  operations;  and  (3)  conduct  a  de¬ 
tailed  analysis  to  examine  the  long-range  improvements  needed  to 
modernize  computer  operations  at  air  bases.  The  Air  Force  pro¬ 
jects  that  total  acquisition  and  operation  savings  will  be 
§1.8  billion  over  the  next  20  years. 

In  reporting  on  the  status  of  the  program  in  March  1983,1 
we  stated  that  the  Air  Force  had  selected  a  contractor  and  had 
agreed  to  buy  about  155  new  computer  systems,  even  though  impor¬ 
tant  software  conversion  from  the  old  to  the  new  computers  and 
performance  testing  had  not  been  completed. 
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During  this  followup,  conducted  from  January  to  early  Sep¬ 
tember  1983,  we  reviewed  the  Air  Force's  progress  in  completing 
the  software  conversion  and  testing  and  the  steps  taken  to  re¬ 
duce  the  costs  and  increase  the  effectiveness  of  the  Phase  IV 
program.  Just  prior  to  issuing  this  report,  we  updated  the  sta¬ 
tus  of  these  efforts.  We  did  not  validate  the  cost  data  pro¬ 
vided  by  the  Air  Force. 

Our  findings  are  summarized  below  and  are  discussed  in  de¬ 
tail,  along  with  our  objectives,  scope,  and  methodology,  in 
appendix  I. 

SOFTWARE  CONVERSION  AND  TESTING 
PROBLEMS  DELAY  IMPLEMENTATION 


The  Air  Force  and  the  contractor  have  taken  steps  to  mini¬ 
mize  delays  and  to  complete  software  conversion  and  testing  ef¬ 
forts.  They  have  made  several  changes  in  software  conversion 
and  testing  schedules,  combined  various  software  conversion  and 
testing  efforts,  and  held  frequent  meetings  to  identify  and  re¬ 
solve  problems.  However,  they  continue  to  experience  difficul¬ 
ties  in  (1)  converting  software  from  old  to  new  computers  and 
(2)  demonstrating  that  the  computers  operate  properly  and  can 
handle  the  expected  workloads  at  the  bases. 

In  December  1980,  when  the  initial  software  conversion  ef¬ 
forts  started  on  Phase  IV,  the  Air  Force  planned  to  have  a  fully 
operational  standard  base  supply  system  at  its  first  site — 
Langley  Air  Force  Base  in  Virginia — by  August  1,  1983.  Follow¬ 
ing  this  would  be  a  phased  implementation  of  the  supply  system 
onto  the  new  computers  at  the  remaining  117  air  bases.  However, 
the  Air  Force  has  missed  target  dates  for  completing  converted 
software  validation  tests  and  now  does  not  expect  to  implement  a 
fully  operational  system  at  Langley  until  February  1984.  Ac¬ 
cording  to  the  Air  Force,  this  6-month  delay  will  increase  Phase 
IV  program  costs  by  about  $6  million  and  will  prevent  the  Air 
Force  from  starting  worldwide  implementation  of  the  supply 
system  until  May  1,  1984. 

Program  management  officials  believe  that  the  Air  Force  can 
recover  from  these  delays  over  an  11-month  period  by  installing 
computers  at  up  to  21  sites  per  month.  The  officials  said,  how¬ 
ever,  that  they  have  not  obtained  commitments  from  the  contrac¬ 
tor  or  major  commands  to  further  accelerate  the  implementation 
schedule — which  has  already  been  increased  from  10  to  16  instal¬ 
lations  per  month  due  to  previous  delays. 

In  addition  to  completing  the  standard  base  supply  system, 
the  Air  Force  and  the  contractor  must  convert  more  than  300 
other  software  systems  from  old  to  new  computers.  Program  man¬ 
agement  officials  said  that  these  conversions  are  proceeding  on, 
or  ahead  of,  schedule.  However,  if  problems  persist  with  the 
supply  system,  or  if  the  accelerated  implementation  schedule 
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cannot  be  attained,  or  if  difficulties  arise  during  the  remain¬ 
ing  software  conversions,  final  Phase  IV  implementation,  sched¬ 
uled  for  completion  by  July  1985,  could  be  delayed.  The  Air 
Force  has  estimated  that  if  this  occurs.  Phase  IV  program  costs 
will  increase  by  about  $10  million  for  each  month  of  delay. 

Along  with  the  delay  in  completing  the  first  operational 
supply  system,  the  Air  Force  has  been  unable  to  complete  the 
workload  test  for  the  new  computers.  Consequently,  the  Air 
Force  does  not  know  if  the  proposed  system  hardware  will  be 
technically  and  operationally  adequate  to  meet  the  bases'  ex¬ 
pected  workloads. 

To  prevent  any  further  delays,  we  believe  that  the  Air 
Force  must  continue  to  closely  monitor  this  complex,  multi- 
billion-dollar  program.  Although  every  effort  should  be  made  to 
stay  on  schedule  and  to  avoid  increased  program  costs,  overall 
program  quality  should  not  be  compromised.  The  Air  Force  must 
insist  on  the  software  being  completely  converted  and  thoroughly 
tested  at  Langley  before  being  implemented  at  other  bases.  This 
approach  and  a  full  postevaluation  are  necessary  to  ensure  that 
system  performance  is  acceptable  for  worldwide  implementation. 
(See  app.  I,  pp.  3  to  7.) 

PROGRAM  COSTS  HAVE  BEEN  REDUCED, 

BUT  MORE  CAN  BE  DONE 

In  meeting  specific  commitments  made  to  your  committee  in 
March  1980  regarding  Phase  IV,  the  Air  Force  has 

— reduced  the  number  of  computers  to  be  acquired  from  227 
to  155, 

— identified  250  personnel  positions  that  could  be  elimi¬ 
nated  by  Phase  IV,  and 

— adopted  a  lease/purchase  approach  for  the  computer  hard¬ 
ware  that  will  save  the  government  about  $690  million. 

These  efforts  are  commendable,  but  we  believe  that  the  Air 
Force  can  further  reduce  costs  by 

— installing  a  multiprocessor  system, 2  where  cost  effec¬ 
tive,  instead  of  two  separate  systems; 

— establishing  additional  regional  centers,  as  well  as 
linking  more  air  bases  to  existing  centers,  to  take  full 
advantage  of  opportunities  to  share  computer  resources; 


2a  multiprocessor  system  is  a  computer  using  two  or  more  process¬ 
ing  units  that  have  access  to  a  common  main  storage  in  which  data 
can  be  entered,  retained,  and  retrieved. 


3 


B-211087 


—conducting  a  detailed  analysis  of  staffing  requirements 
and  ensuring  that  staffing  is  kept  to  a  minimum;  and 

—implementing  a  formal  system  to  track  the  leasing  and 
purchasing  of  computer  equipment. 

The  Assistant  Secretary  of  the  Air  Force  (Financial  Manage¬ 
ment)  agreed  that  close  monitoring  of  program  costs  is  needed. 
(See  app.  I,  pp.  8  and  9.) 

AIR  FORCE  EFFORTS  TO  IMPROVE  DAY-TO-DAY 
OPERATIONS  AND  COMPLETE  SYSTEM  ANALYSIS 

The  Air  Force  has  met  its  commitment  to  improve  day-to-day 
operations  of  computers  by  (1)  reducing  the  use  of  punched 
cards/  (2)  using  visual  display  terminals  rather  than  printing 
terminals,  and  (3)  increasing  the  use  of  data  base  management 
and  generalized  retrieval  systems.  The  Air  Force  expects  even 
more  improvements  once  its  base  level  systems  are  redesigned  to 
better  meet  user  needs. 

Regarding  the  Air  Force's  commitment  to  conduct  a  detailed 
analysis  of  the  bases'  systems,  the  Assistant  Secretary  of  the 
Air  Force  (Financial  Management)  said  that  the  analysis  will  not 
be  completed  until  after  Phase  IV  has  been  implemented  world¬ 
wide.  According  to  the  Assistant  Secretary,  program  officials 
have  been  prevented  from  starting  the  analysis  sooner  because  of 
organizational  difficulties  and  because  guidelines  for  perform¬ 
ing  the  analysis  have  been  unclear.  Consequently,  this  work  is 
several  years  behind  the  schedule  established  by  the  Air  Force 
in  early  1980. 

At  our  suggestion,  the  Assistant  Secretary  agreed  to  re¬ 
affirm  top  management *8  consnitment  to  completing  the  analysis 
and  to  establish  a  formal  charter  with  clear  directives  author¬ 
izing  the  resources  needed  for  the  analysis.  (See  app.  I, 

p.  10.) 

FOLLOWUP  ACTIONS  PLANNED 


The  Assistant  Secretary  of  the  Air  Force  (Financial  Manage¬ 
ment)  is  acting  on  the  issues  raised  in  our  report.  Also,  the 
Secretary  of  the  Air  Force  has  directed  the  Air  Force  Audit 
Agency  and  the  Inspector  General  to  follow  the  Phase  IV  program 
and  to  report  the  results  of  their  reviews  to  the  Assistant 
Secretary  of  the  Air  Force  (Financial  Management) .  Copies  of 
their  reports  are  to  be  sent  to  us  and  to  your  committee. 


We  did  not  obtain  official  comments  on  this  report  from 
the  agency  or  the  contractor.  However,  we  did  discuss  the 
report  contents  with  program  officials  and  have  included  their 
comments  where  appropriate. 
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Unless  you  release  its  contents  earlier,  we  plan  no 
further  distribution  of  this  report  until  30  days  after  its 
issue  date.  The  ,,  we  will  send  copies  to  the  Secretaries  of 
Defense  and  the  Air  Force  and  to  other  interested  parties  upon 
request. 


Sincerely  yours, 


of  the  United  States 
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AIR  FORCE  PROGRESS  IN  IMPLEMENTING  THE  PHASE  IV 
BASE  LEVEL  COMPUTER  REPLACEMENT  PROGRAM 

BACKGROUND 

In  April  1976  the  Air  Force  started  the  Phase  IV  program  to 
replace  existing  base  level  computer  systems  at  118  air  bases 
around  the  world.  The  program  objectives  are  to  provide 

(1)  cost  effective,  responsive,  and  reliable  computer  support 
for  a  variety  of  base  level  supply,  administrative,  and  other 
functions;  (2)  a  safe  transition  of  current  applications  soft¬ 
ware  to  new  computers;  and  (3)  flexibility  to  grow.  Over  the 
next  20  years.  Phase  IV  acquisition  and  operation  costs  will 
total  about  $5  billion. 

When  it  released  its  request  for  proposals  in  December 
1978,  the  Air  Force  planned  to  buy  two  computer  systems  for  most 
of  its  bases — or  a  total  of  227  computers  for  118  bases.  At 
your  request,  we  started  a  review  of  Phase  IV  in  March  1979. 

Soon  after,  at  your  committee's  October  10,  1979,  hearings,  we 
testified  that  the  Air  Force's  plan  to  buy  two  computers  for 
most  bases  was  unnecessarily  expensive  and  restricted  competi¬ 
tion.  We  restated  our  concerns  in  an  October  1979’  report  and 
recommended  that  the  Air  Force  cancel  its  plans  to  buy  two  com¬ 
puters  for  most  bases  and  to  revise  its  request  for  proposals  to 
better  reflect  functional  requirements. 

In  response  to  congressional  hearings,  our  report,  and  a 
report  by  the  House  Committee  on  Government  Operations,2  the 
Air  Force,  in  March  1980,  pledged  to 

— reduce  the  number  of  computers  to  be  acquired  and  the 
costs  of  computer  equipment,  personnel,  and  construction; 

— significantly  improve  the  day-to-day  operation  of  base 
level  systems;  and 

— conduct  a  detailed  functional  analysis  of  base  level 
systems. 

The  Air  Force  modified  its  request  for  proposals  to  reflect 
this  redirection  of  Phase  IV  and,  in  December  1980,  awarded 
fixed-price  contracts  to  Sperry  Corporation  and  Burroughs  Cor¬ 
poration.  under  the  Air  Force's  acquisition  strategy,  the  two 
contractors  were  responsible  for  (1)  converting  software  for  13 
of  the  Air  Force's  largest  and  most  complex  computer  systems  and 

(2)  demonstrating  the  converted  software  on  their  proposed 


^The  Air  Force  Should  Cancel  Plans  To  Acquire  Two  Computer  Systems 
At  Most  Bases  ( FGMSD-80-1 5 ,  Oct.  26.  1979). 

2The  Department  of  the  Air  Force's  Phase  IV  Program  Should  Be  Re¬ 
directed  (H.  Rept.  96-694,  Dec.  10.  19^9). 
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equipment.  The  purpose  of  this  strategy  was  to  enable  the  Air 
Force  to  fully  evaluate  the  contractors'  price  proposals  and 
technical  capabilities  before  awarding  the  production  contract 
to  one  of  them.  After  the  corporations  completed  the  testing 
and  demonstrations  and  submitted  final  price  proposals  for  the 
contract,  the  Air  Force  was  to  choose  a  contractor  on  the  basis 
of  this  comprehensive  evaluation,  with  cost  being  the  primary 
factor  considered. 

Even  though  the  two  corporations  had  not  completed  impor¬ 
tant  conversion  and  testing  efforts,  the  Air  Force,  on  January 
27,  1983,  awarded  an  8-year,  $476.2  million  contract  to  Sperry. 
This  selection  (or  "production  buy  decision,"  as  it  is  termed) 
committed  the  Air  Force  to  buy  about  155  computer  systems  from 
Sperry. 

In  March  19833  we  reported  that  Air  Force  program  manage¬ 
ment  officials  said  they  had  proceeded  with  the  production  buy 
decision  because  (1)  both  companies  had  demonstrated  adequate 
technical  capabilities  to  meet  program  requirements  and  (2) 
Sperry's  final  price  proposals  were  significantly  lower  than 
Burroughs'.  The  officials  said  that  the  Air  Force  could  join 
with  the  selected  contractor  in  solving  problems  and  in  complet¬ 
ing  the  remaining  conversion  and  testing.  They  added  that  the 
selected  contractor  would  have  to  complete  all  software  conver¬ 
sion  and  testing  and  correct  all  deficiencies  before  the  Air 
Force  implemented  the  new  systems  worldwide.  When  it  made  its 
production  buy  decision  in  January,  the  Air  Force  planned  to 
have  an  operational  standard  base  supply  system  at  Langley  Air 
Force  Base  in  Virginia  by  August  1,  1983. 

OBJECTIVES,  SCOPE,  AND  METHODOLOGY 

In  responding  to  your  January  13,  1983,  request,  we  sought 
to  review  (1)  the  Air  Force's  efforts  (a)  to  complete  software 
conversion  and  testing  and  (b)  to  ensure  that  Phase  IV  computers 
performed  acceptably  before  being  implemented  worldwide,  and  (2) 
the  Air  Force's  progress  toward  fulfilling  prior  commitments 
made  to  your  committee. 

To  learn  what  actions  the  Air  Force  plans  to  take  to  ensure 
the  program's  success,  we  discussed  review  issues  in  detail  with 
the  Assistant  Secretary  of  the  Air  Force  (Financial  Management), 
the  senior  program  official  and  source  selection  authority  for 
the  Phase  IV  acquisition.  We  visited  Air  Force  headquarters  in 
Washington,  D.C.,  and  key  offices  involved  in  managing  the  Phase 
IV  program — Automated  Systems  Project  Office,  Data  Systems 
Design  Center,  Systems  Evaluation  Center,  and  Operational  Test 
and  Evaluation  Center,  all  located  at  Gunter  Air  Force  Station, 
Alabama.  From  January  to  early  September  1983,  we  interviewed 
personnel  at  these  offices  and  reviewed  pertinent  documents 
provided  by  the  Air  Force.  We  also  interviewed  personnel  at  key 


3Status  of  the  "Phase  IV"  Base  Level  Computer  Replacement  Program 
(GAO/AFMD-83-58,  Mar.  16,  1983). 
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major  commands  and  air  bases  and  representatives  of  Sperry  Cor¬ 
poration.  We  updated  the  status  of  the  Air  Force  actions  just 
prior  to  issuing  this  report. 

We  performed  our  review  in  accordance  with  generally  ac¬ 
cepted  government  audit  standards.  We  did  not  obtain  official 
agency  or  contractor  comments  on  this  report.  We  also  did  not 
validate  the  cost  data  provided  by  the  Air  Force. 

AIR  FORCE  ACTIONS  TO  COMPLETE 
SOFTWARE  CONVERSION  AND  TESTING 


Since  making  the  production  buy  decision,  the  Air  Force  has 
joined  with  the  contractor  in  working  toward  resolving  the  man¬ 
agement  and  technical  problems  encountered  in  the  Phase  IV  pro¬ 
gram.  However,  the  Air  Force  and  the  contractor  have  continued 
to  experience  difficulties  in  converting  some  of  the  large,  com¬ 
plex  software  systems  and  in  completing  the  tests  to  demonstrate 
that  the  systems  operate  properly  and  can  meet  base  level  work¬ 
loads.  The  Air  Force  has  missed  target  dates  for  completing 
these  tests  and  is  now  about  6  months  behind  schedule  in  imple¬ 
menting  a  fully  operational  standard  base  supply  system  at  the 
first  site — Langley  Air  Force  Base.  Delays  in  completing  the 
software  conversion  and  testing  at  Langley  will  also  delay  com¬ 
pletion  of  the  workload  test  and  the  start  of  program  implemen¬ 
tation  at  other  bases  worldwide .  The  Air  Force  estimates  that 
these  delays  will  increase  program  costs  by  about  $6  million. 

Remaining  implementation  efforts  are  far  from  complete, 
since  the  Air  Force  and  the  contractor  still  have  to  convert  300 
other  software  systems.  To  make  certain  that  all  conversions 
are  successful,  the  Air  Force  must  closely  monitor  the  Phase  IV 
program. 

Software  conversion  and 
testing  difficulties  continue 

When  the  contractor  started  the  software  conversion  efforts 
in  December  1980,  the  Air  Force  planned  to  have  a  fully  opera¬ 
tional  supply  system  at  Langley  by  August  1,  1983.  Validation 
tests,  which  verify  that  the  Langley  system  and  the  software  up¬ 
date  have  been  correctly  converted,  were  originally  scheduled 
for  completion  in  August  1982  and  subsequently  rescheduled  for 
completion  prior  to  the  production  buy  decision  in  January 
1983.  An  evaluation  of  the  converted  systems  by  the  primary 
user  organization  was  also  expected  to  be  completed  by  that 
time.  Validation  testing  of  the  updated  system  was  to  be  com¬ 
pleted  by  April  15,  1983.  The  Air  Force  missed  these  important 
target  dates  and  decided  to  consolidate  and  combine  these  vali¬ 
dation  and  evaluation  efforts  with  the  implementation  and  con¬ 
version  testing  at  Langley. 

When  we  completed  our  field  work  in  early  September  1983, 
the  contractor  had  completed  validation  testing  on  153  of  the 
440  supply  system  programs.  At  that  time,  the  Air  Force  was 
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expecting  immediate  delivery  of  the  remaining  287  programs  and 
had  plans  to  complete  the  system  implementation  and  conversion 
at  Langley  by  November  30,  1983.  Just  prior  to  issuing  this 
report,  we  checked  on  the  status  of  these  efforts  and  learned 
that  the  Langley  system  has  experienced  additional  delays  and  is 
not  expected  to  be  completed  until  February  1984.  According  to 
Air  Force  program  management  officials,  this  additional  delay 
occurred  because  a  key  on-line  software  program  for  processing 
and  interfacing  supply  transactions  could  not  meet  performance 
requirements  and  had  to  be  replaced  with  a  faster  and  more  effi¬ 
cient  program  capable  of  handling  the  supply  systems’  high- 
volume  workload. 

The  workload  test  was  also  incomplete  when  we  finished  the 
field  work.  Because  of  the  additional  supply  system  delays, 
this  important  test  is  not  expected  to  be  completed  until  Febru¬ 
ary  20,  1984 — 18  months  after  the  originally  scheduled  target 
date  of  September  1 ,  1982 .  The  contractor ' s  new  computer  has 
been  unable  to  process  the  overall  workload  as  fast  as  the  ex¬ 
isting  computer,  and  36  deficiencies  remain  unresolved.  The  Air 
Force  concedes  that  the  major  risk  associated  with  the  workload 
test  is  that  proposed  hardware  configurations  may  not  satisfac¬ 
torily  fulfill  the  performance  requirements.  If  any  of  the  re¬ 
maining  configurations  to  be  tested  lead  to  unsatisfactory  per¬ 
formance  that  cannot  be  corrected,  additional  hardware  may  be 
required.  Except  for  potential  computer  memory  size  increases, 
the  contractor  is  committed  to  completing  the  workload  test  on 
the  proposed  hardware  configurations.  However,  until  the  test 
is  completed,  the  Air  Force  cannot  positively  determine  that  the 
contractor's  proposed  hardware  configurations  will  be  technical¬ 
ly  and  operationally  adequate  for  meeting  base  level  workloads. 

The  Assistant  Secretary  of  the  Air  Force  (Financial  Man¬ 
agement)  said  the  Air  Force  will  continue  testing  the  software 
until  the  systems  meet  all  requirements  and  are  fully  capable  of 
supporting  base  level  workloads.  He  said  that  all  remaining 
tests,  including  the  workload  test,  will  be  completed  before  the 
Air  Force  accepts  the  system,  and  that  the  contractor  will  be 
required  to  correct  software  and  system  performance  deficiencies 
without  increasing  the  size  of  hardware  configurations  agreed 
upon  in  the  Phase  IV  contract. 

In  May  1983,  the  Air  Force  Operational  Test  and  Evaluation 
Center  completed  its  test  report  on  the  Qualification  Operation¬ 
al  Test  and  Evaluation.  This  independent  test  consisted  of  an 
Air  Force  hands-on  test  of  the  contractor's  computer  system  and 
was  designed  to  evaluate  system  performance  and  reliability  as 
well  as  other  factors,  such  as  computer  operator  staffing 
requirements. 

The  test  report  stated  that  (1)  system  performance  was  de¬ 
ficient  and  was  not  capable  of  meeting  base  level  data  proces¬ 
sing  requirements  as  well  as  they  are  currently  being  met,  (2) 
processing  was  slow  and  products  generated  by  the  system  were 
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incomplete,  and  (3)  the  proposed  hardware  configuration  may  not 
be  adequate. 


The  report  attributed  the  system's  poor  performance  to  the 
design  of  certain  software  systems  and  to  numerous  software  er¬ 
rors  and  deficiencies.  Air  Force  program  management  officials 
said  that  many  of  these  errors  and  deficiencies  were  caused  be¬ 
cause  the  validated  software  for  some  of  the  systems,  particu¬ 
larly  the  supply  system,  was  not  available.  They  said  that  both 
software  and  system  performance  deficiencies  would  be  corrected 
before  final  implementation  and  conversion  at  Langley.  The  of¬ 
ficials  also  said  that  the  deficiencies  would  be  resolved  with¬ 
out  increasing  the  size  of  hardware  configurations. 


In  our  March  1983  report,  we  pointed  out  that  because  the 
Operational  Test  and  Evaluation  Center  had  not  completed  its 
test,  the  Air  Force  could  not  verify  the  actual  staffing  re¬ 
quired  for  the  contractor's  computer  configurations.  While  the 
completed  test  report  concluded  that  staffing  requir 
satisfactory,  it  stated  that  further  measurement  wa 
establish  actual  staffing  requirements,  since  the  r 
sions  were  based  on  a  test  environment.  The  Assist 
said  that  the  Air  Force  would  continually  monitor  t 
ensure  that  staffing  levels  were  kept  to  a  minimum. 


jnts  were 
leeded  to 
•rt  conclu- 
Secretary 
orogram  to 


Delays  have  increased  program  costs 

Sperry  agreed  to  complete  the  tests  that  were  not  performed 
within  the  initial  contract  period  at  no  additional  cost  to  the 
government.  Because  the  Air  Force  has  joined  with  the  contrac¬ 
tor  in  completing  the  tests,  the  Air  Force  has  expended  over 
2,000  additional  staff  days  and  $410,000  in  travel  and  per  diem 
costs  in  the  first  4  months  after  the  production  buy  decision. 
Program  costs  will  continue  to  increase  with  each  additional 
month  the  testing  efforts  are  continued. 

Not  implementing  an  operational  supply  system  at  Langley  by 
August  1,  as  planned,  has  delayed  the  start  of  the  program  at 
major  commands  and  other  bases  worldwide  and  has  also  increased 
program  costs.  An  Air  Force  contingency  plan,  issued  in  Septem¬ 
ber  1983,  indicated  that  the  Air  Force  could  recover  from  the 
delay  in  7  months  by  installing  the  computers  at  up  to  16  sites 
per  month.  The  contractor  had  agreed  to  16  installations  per 
month  and  Air  Force  commands  had  reported  they  could  provide 
sufficient  resources  to  meet  the  accelerated  schedule  and  the 
additional  workload.  Previously,  however,  the  Air  Force  had 
said  it  could  not  support  any  more  than  10  installations  a  month 
due  to  limited  resources. 


The  recent  supply  system  delay  reported  to  us  in  late  No¬ 
vember  1983  means  the  Air  Force  will  not  be  able  to  start  world¬ 
wide  implementation  of  Phase  IV  until  May  1,  1984.  According  to 
program  management  officials,  the  Air  Force  can  still  recover 
from  the  delay  in  11  months  by  installing  computers  at  up  to  21 
sites  per  month.  The  officials  said,  however,  that  they  have 
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not  obtained  commitments  from  the  contractor  or  the  major  com¬ 
mands  to  further  accelerate  the  implementation  schedule. 

According  to  the  Air  Force,  the  6-month  delay — from  August 
1983  to  February  1984 — in  completing  the  Langley  system  will  in¬ 
crease  program  costs  by  approximately  $6  million.  The  Air  Force 
also  said  that  program  costs  could  increase  even  more  if  an  ac¬ 
celerated  implementation  schedule  cannot  be  met. 

Remaining  implementation  efforts 
are  far  from  complete 

In  addition  to  completing  the  software  conversion  of  the  13 
major  systems  started  before  the  production  buy  decision,  the 
Air  Force  and  the  contractor  have  to  convert  more  than  300  other 
software  systems,  which  are  also  processed  on  the  existing  com¬ 
puters.  During  this  secondary  conversion  effort,  the  contractor 
is  required  to  convert  eight  standard  Air  Force  systems  contain¬ 
ing  about  358,000  lines  of  code  (or  program  instruction).  The 
Air  Force  Data  Systems  Design  Center  must  convert  21  standard 
systems  containing  about  550,000  lines  of  code,  and  several  ma¬ 
jor  commands  have  to  convert  a  total  of  285  command-unique  sys¬ 
tems  with  about  2.7  million  lines  of  code. 

The  Air  Force  plans  to  complete  this  large  software  conver¬ 
sion  effort,  integrate  all  the  systems  on  a  single  hardware  con¬ 
figuration,  test  overall  performance,  and  install  a  fully  opera¬ 
tional  system  at  Langley  by  July  1984.  The  Air  Force's  current 
plan  calls  for  installing  the  last  of  the  Phase  IV  computers 
worldwide  by  July  1985. 

Although  converting  the  initial  13  systems  has  been  more 
difficult  than  expected,  the  Air  Force  loss  not  anticipate  as 
much  difficulty  in  converting  these  other  systems  because  most 
are  smaller  and — in  contrast  to  those  programs  already  con¬ 
verted — are  already  written  in  a  higher  level  programming 
language . 

Air  Force  program  management  officials  told  us  that  these 
efforts  are  proceeding  on  or  ahead  of  schedule  and,  thus  far, 
have  not  been  affected  by  the  supply  system  delays  at  Langley. 
However,  if  problems  persist  with  the  supply  system,  if  an  ac¬ 
celerated  implementation  schedule  cannot  be  attained,  or  if  dif¬ 
ficulties  arise  during  the  remaining  software  conversions,  the 
1985  Phase  IV  implementation  schedule  could  be  delayed.  The  Air 
Force  has  estimated  that  a  1-month  delay  in  final  Phase  IV  im¬ 
plementation  would  increase  program  costs  by  $10  million;  a 
3-month  slip  in  the  overall  schedule  would  cost  $33  million. 

Air  Force  efforts  to  correct  problems 

Both  Air  Force  and  contractor  program  management  officials 
have  expressed  general  satisfaction  with  the  program's  current 
status.  Pointing  out  that  the  initial  software  conversion 
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effort  involved  the  13  largest  and  most  complex  Air  Force 
standard  systems  containing  about  1.6  million  lines  of  code, 
officials  believe  that  the  program  has  proceeded  about  as  well 
as  could  be  expected.  They  said  implementation  has  been  diffi¬ 
cult  due  to  (1)  problems  in  converting  from  machine  language  to 
a  high  order  programming  language,  (2)  inadequate  documentation 
of  existing  systems,  and  (3)  a  cumbersome  and  time  consuming 
test  process . 

The  Air  Force  and  the  contractor  have  taken  several  steps 
to  minimize  program  delays  and  to  complete  the  software  conver¬ 
sion  and  testing  efforts.  For  example,  both  have  agreed  to 
several  changes  in  software  conversion  and  testing  schedules  to 
speed  the  process.  The  contractor  was  to  deliver  two  separate 
updates  to  the  initial  software  systems,  but  these  were  combined 
to  accelerate  delivery  of  the  final  package.  Similarly,  the 
original  package  and  update  for  each  of  the  follow-on  systems 
were  combined.  In  addition,  the  Air  Force  provided  technical 
representatives  at  test  sites  to  aid  the  contractor  in  identify¬ 
ing  and  solving  problems.  More  frequent  meetings  were  scheduled 
between  the  Air  Force  and  contractor  program  managers  to  facili¬ 
tate  the  exchange  of  information.  These  meetings  were  held  to 
(1)  identify  potential  problems,  (2)  solve  the  problems,  and  (3) 
identify  alternative  methods  to  satisfy  contract  requirements 
more  efficiently. 

The  Air  Force  developed  a  project  management  system  to 
track  contractor  progress,  including  analyses  of  schedules,  per¬ 
formance,  and  costs.  When  contractor  performance  became  delin¬ 
quent  during  the  transition  period,  the  Air  Force  issued  delin¬ 
quency  notices  and  suspended  progress  payments.  In  response, 
the  contractor  increased  its  staff,  installed  more  equipment, 
and  leased  facilities  closer  to  the  Langley  test  site.  The  con¬ 
tractor  also  reorganized  its  internal  management  and  reporting 
structures  to  focus  more  attention  on  the  Phase  IV  program. 

Air  Force  should  continue  to 
closely  monitor  Phase  IV  program 

To  prevent  any  further  delays,  we  believe  that  the  Air 
Force  must  continue  to  closely  monitor  this  complex,  multi- 
billion-dollar  program.  Although  every  effort  should  be  made  to 
stay  on  schedule  and  to  avoid  increased  program  costs,  overall 
program  quality  should  not  be  compromised.  The  Air  Force  must 
insist  on  the  software  being  completely  converted  and  thoroughly 
tested  at  Langley  before  being  implemented  at  other  bases.  This 
approach,  and  a  full  postevaluation,  are  necessary  to  ensure 
that  system  performance  is  acceptable  for  worldwide  implementa¬ 
tion.  The  Assistant  Secretary  of  the  Air  Force  (Financial  Man¬ 
agement)  said  that  he  intends  to  take  all  actions  needed  to  en¬ 
sure  that  costs  are  held  to  a  minimum,  that  schedules  are  met, 
and  that  program  quality  remains  high. 
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REDIRECTION  OF  PHASE  IV  REDUCED  COST, 
BUT  ADDITIONAL  STEPS  ARE  NEEDED 


The  redirection  of  Phase  IV  has  produced  significant  sav¬ 
ings.  In  March  1980,  the  Secretary  of  the  Air  Force  reported 
that  approximately  $800  millic..  in  costs  could  be  avoided  as  a 
result  of  the  program  redirection.  As  a  result  of  this  redirec¬ 
tion  and  other  Air  Force  actions,  the  Air  Force  now  estimates 
that  total  acquisition  and  operation  savings  for  Phase  IV  will 
be  $1.8  billion  over  the  next  20  years. 

To  meet  its  specific  commitments  to  your  committee,  the  Air 
Force  has  reduced  the  number  of  Phase  IV  computer  systems  to  be 
acquired  from  227  to  155  by  installing  a  single  computer  system 
at  most  air  bases  and  by  establishing  14  regional  centers.  We 
believe  that  further  reductions  can  be  made  by  (1)  installing 
multiprocessor  systems,  where  cost  effective,  at  locations  cur¬ 
rently  scheduled  to  receive  two  separate  computer  systems  and 
(2)  taking  full  advantage  of  regionalization  to  increase  sharing 
of  computer  resources . 

The  Air  Force  plans  to  install  two  separate  computer  sys¬ 
tems  at  six  aerial  port  bases  to  handle  expected  workload  surges 
and  at  11  of  the  14  regional  centers  which  will  be  required  to 
support  one  or  more  remote  bases.  On  the  basis  of  our  analysis 
of  contractor  equipment  specifications  and  Air  Force  studies  on 
equipment  and  personnel  requirements,  we  estimate  that  using  a 
single  multiprocessor  system  instead  of  two  separate  systems  at 
these  locations  would  save  up  to  $16.9  million  over  8  years. 

The  Air  Force  also  may  be  able  to  use  single  systems  at  some  of 
its  24  overseas  bases  that  are  scheduled  to  receive  two  separate 
systems . 

A  multiprocessor  system  can  be  sized  to  provide  essentially 
the  same  capability  as  two  separate  systems  and  can  provide 
extra  capacity  for  workload  surges  as  well.  With  a  single 
multiprocessor  system,  the  Air  Force  could  eliminate  up  to  five 
computer  operator  positions  and  the  requirement  for  a  second 
front-end  communications  processor  at  each  location.  The  Phase 
IV  program  manager  agreed  that  the  Air  Force  might  be  able  to 
use  multiprocessor  systems  at  some  of  these  locations,  but  he 
said  that  all  technical  aspects  must  first  be  evaluated  by  hard¬ 
ware  and  software  engineers. 

Considering  the  advanced  technology  of  the  new  computer 
systems,  we  believe  that  the  Air  Force  could  have  realized 
greater  savings  by  establishing  more  regional  centers.  The  Air 
Force  was  able  to  reduce  facilities  construction  costs  by  estab¬ 
lishing  the  14  regional  centers  and  consolidating  data  proces¬ 
sing  facilities  at  most  locations.  However,  by  establishing 
criteria,  such  as  a  maximum  100-mile  distance  and  2-hour  driving 
time  between  air  bases  considered  for  regionalization,  the  Air 
Force  has  missed  additional  savings  opportunities. 
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Some  sites  were  rejected  and  not  fully  evaluated  because 
they  exceeded  those  limits.  Other  sites  were  rejected  because 
major  commands  would  not  agree  to  having  their  bases  with  opera¬ 
tional  flying  units  supported  by  a  remote  base's  computer 
system.  In  analyzing  the  Air  Force's  regionalization  studies, 
and  by  including  estimates  of  personnel  savings,  we  identified 
five  locations  that  would  have  been  cost-effective  regional  cen¬ 
ters  and  three  additional  air  bases  that  could  have  been  aligned 
with  the  14  regional  centers.  The  Air  Force  rejected  these  al¬ 
ternatives  because  the  estimated  savings  were  uncertain  before 
contract  award,  and  adding  bases  would  have  increased  the  risks 
of  delaying  the  program.  However,  Air  Force  officials  said  that 
they  plan  to  further  examine  regionalization  issues  and  alterna¬ 
tives,  including  the  feasibility  and  cost  effectiveness  of 
establishing  additional  regional  centers,  as  well  as  linking 
more  bases  to  existing  centers. 

The  Air  Force  plans  to  eliminate  at  least  250  personnel 
positions  with  the  Phase  IV  program?  however,  these  reductions 
are  not  expected  to  take  place  until  after  program  implementa¬ 
tion  is  completed  in  1985.  Following  the  1980  redirection  of 
the  Phase  IV  program,  the  Air  Force  identified  the  positions  to 
be  eliminated  and  established  a  baseline  to  account  for  the  per¬ 
sonnel  reductions.  Only  base  level  computer  operations  posi¬ 
tions  were  considered  in  establishing  the  baseline,  and  all  the 
reductions  identified  were  in  the  supply  area.  We  found  that 
the  Air  Force  plans  to  use  the  positions  eliminated  by  Phase  IV 
to  meet  personnel  requirements  outside  the  base  level  computer 
operations  area.  The  Air  Force  may  not  realize  any  actual 
reductions  because  staffing  is  expected  to  increase  with  work¬ 
load  changes  and  new  programs  and  missions. 

The  Air  Force  has  adopted  a  lease/purchase  approach  for  the 
computer  hardware,  which  will  lead  to  substantial  savings. 

Using  this  approach,  the  Air  Force  will  lease  the  hardware  for 
about  9  to  16  months,  and  then  purchase  the  equipment  with 
credit  for  all  lease  payments  made.  To  save  the  most  money,  the 
Air  Force  must  carefully  monitor  the  lease/purchase  credits  so 
that  it  can  identify  the  best  time  to  buy  the  numerous  equipment 
items,  which  have  varying  lease  and  purchase  prices.  Compared 
to  original  life-cycle  program  estimates,  the  Air  Force  will 
save  about  $690  million  by  purchasing  instead  of  leasing  the 
equipment. 

The  Assistant  Secretary  of  the  Air  Force  (Financial  Manage¬ 
ment)  said  that  close  monitoring  of  the  program  cost  is  needed 
and  that  further  savings  can  be  achieved.  At  our  suggestion,  he 
agreed  to: 

— evaluate  the  use  of  multiprocessor  systems  at  locations 
scheduled  to  receive  two  separate  systems, 

— reexamine  the  regionalization  alternatives  and  issues  and 
take  full  advantage  of  opportunities  to  share  computer 
resources. 


9 


APPENDIX  I 


APPENDIX  I 


— conduct  a  detailed  analysis  of  staffing  requirements  and 
ensure  base  level  computer  operations  staffing  is  kept  to 
a  minimum,  and 

— implement  a  formal  system  to  track  the  lease  and  purchase 
of  computer  equipment. 

PROGRESS  TOWARD  MODERNIZING 
BASE  LEVEL  SYSTEMS  AND  COM¬ 
PLETING  THE  FUNCTIONAL  ANALYSIS 

In  1980,  the  Air  Force  assured  your  committee  that  Phase  IV 
would  significantly  improve  day-to-day  operations  of  computer 
systems.  The  Air  Force  also  promised  to  conduct  a  detailed 
functional  analysis  of  these  systems,  examining  the  long-range 
improvements  needed  to  modernize  base  level  operations. 

The  Air  Force  has  made  or  expects  to  make  immediate  im¬ 
provements  in  day-to-day  operations  by  reducing  the  use  of 
punched  cards,  using  visual  display  terminals  rather  than  print¬ 
ing  terminals,  and  increasing  the  use  of  data  base  management 
and  generalized  retrieval  systems.  Air  Force  officials  said 
that  additional  operational  improvements  could  be  achieved  when 
base  level  systems  are  redesigned  or  enhanced  to  better  support 
user  requirements.  The  Air  Force  has  already  identified  numer¬ 
ous  modernization  initiatives  and  has  projected  many  automation 
requirements  involving  enhancements  and  redesign  of  existing  or 
planned  base  level  systems  over  the  next  several  years. 

The  Air  Force  has  begun  a  detailed  analysis  of  base  level 
systems  to  examine  the  information  needs  of  functional  areas  and 
senior  base  managers  in  all  existing  and  planned  manual  and  au¬ 
tomated  systems.  Using  contractor-developed  techniques  and 
methodology,  the  Air  Force  has  initiated  efforts  to  document  2 
of  16  functional  areas  needing  analysis — supply  and  accounting/ 
finance.  Other  important  aspects  of  these  functional  areas  that 
will  be  analyzed  include  (1)  identifying  information  needs  of 
base  level  managers,  (2)  examining  problems  and  inefficiencies, 
and  (3)  determining  interfacing  requirements  for  related  sys¬ 
tems.  The  Air  Force  had  planned  to  begin  this  functional  analy¬ 
sis  in  February  1980  but,  because  of  organizational  difficulties 
and  a  tentative,  unclear  charter  to  authorize  the  work,  did  not 
begin  until  April  1983. 

The  Assistant  Secretary  of  the  Air  Force  (Financial  Manage¬ 
ment)  acknowl  ->d  that  the  Air  Force  has  experienced  organiza¬ 
tional  diffi  -  and  that  guidance  for  performing  the  func¬ 
tional  anal  been  unclear.  At  our  suggestion,  he  agreed 

to  reaffirn  tement's  commitment  and  to  establish  a  for¬ 
mal  charter  ir  directives  for  this  effort.  He  agreed 

that  completi  t<  detailed  functional  analysis  is  essential  to 
making  informed  decisions  on  the  future  of  data  processing  in 
the  Air  Force.  After  completing  the  analysis,  the  Air  Force  ex¬ 
pects  to  implement  many  additional  improvements  made  possible  by 
the  new  Phase  IV  equipment. 
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The  Honorable  Charles  A.  Bowsher 
Comptroller  General 
General  Accounting  Office 
441  G  Street,  N.W. 

Washington,  O.C.  20548 

Dear  General : 

The  Conmlttee  on  Government  Operations  recently  received  an  excellent 
briefing  by  your  staff  on  the  Air  Force's  Base  Level  Automation  Program 
(Phase  IV).  Apparently,  the  project  is  experiencing  serious  management  and 
technical  difficulties  wftich  may  jeopardize  the  successful  completion  of  this 
$5  billion  procurement.  Specifically,  as  of  December  1982.  neither  contractor 
had  completed  all  validation  tests  which  were  scheduled  for  completion  in 
early  August  and  the  performance  by  both  contractors  hrs  been  judged  by  the 
Air  Force  to  be  unsatisfactory.  Further,  the  critical  workload  and  operational 
testing  have  not  been  completed,  even  though  the  Air  Force  plans  to  award 
the  contract  In  February. 

Given  the  size  and  importance  of  this  project,  a  follow-up  review  should 
be  made  by  your  office  to  determine  what  corrective  actions  must  be  taken  to 
remedy  these  serious  deficiencies.  Your  review  shoula  take  Into  consideration 
the  prior  commitments  made  to  this  Committee  by  the  Air  Force  to  reduce  the 
cost  and  increase  the  effectiveness  of  this  procurement.  Since  the  Air  Force 
plans  to  award  this  contract  In  the  near  future*  I  request  an  1nter1.ii  report 
on  this  review  within  60  days  and  a  final  report  within  six  months. 

With  every  good  wish,  1  am 


(913707) 
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